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Abstract- Utilizing fluoroscopy for three-dimensional (3D) in 
vivo analysis is a powerful tool with widespread applications. It 
allows joints (both implanted and normal) to be studied in vivo 
under weight bearing conditions through normal day-to-day 
activities such as walking, leg bending, arm raising, etc. Our 
robust 3D-to-2D registration method allows 3D data to be 
extracted more accurately than similar techniques which employ 
an added preliminary segmentation step. After extracting the 3D 
data, many different joint-specific 3D analyses can be 
performed. This includes automatic calculation of hip 
separation, loci tracking of different joints (e.g. hip, vertebrae, 
shoulder), automatic tracking of ligament attachments, etc. We 
have also extended our 3D-to-2D registration method to 
automatically fit a large number of closely spaced images (e.g. 
200-500), which we term "video fitting". This provides smoother 
motion of the implanted models through a given activity. Also, 
we are able to calculate the contact area between two models. 
This provides contact area information over a range of activities 
for more advanced analysis of the knee joint which could be 
used as input into a finite element model for further analysis. 
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I. Introduction 

Fluoroscopy has emerged as a powerful tool for analyzing 
implanted joints in vivo. The ability to capture data 
dynamically opens up many applications compared to still X- 
rays. This combined with the freedom of motion allowed by 
single plane fluoroscopy allows analysis of many weight 
bearing activities. Komistek et al. have used fluoroscopy to 
perform many distinct 3D analyses including calculation of in 
vivo loading of the knee joint [1], calculation of hip 
separation after total hip arthroplasty [2], and performance 
comparison of posterior stabilized versus posterior cruciate- 
retaining total knee arthroplasty (TKA) [3]. Work has also 
been done using this same 3D-to-2D registration technique to 
analyze normal bones. This includes analysis of normal knees 
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technique utilized in 
s outlined in [6]. As discussed in 
method outperforms similar 
methods which employ an added segmentation step to extract 
a contour around the implant from the original fluoro image. 
Once the 3D information has been extracted from 
fluoroscopy data, a full 3D analysis of the joint is performed. 
This includes calculation of hip separation, loci tracking of 
different joints (e.g. hip, vertebrae, shoulder), and automatic 
tracking of ligament attachments. Recently another tool has 
been added, which we term "video fitting". Instead of 
analyzing a small number of images for a given activity (e.g. 
capturing a fluoro image every 10° for a deep knee bend 
(DKB)), the activity is analyzed using a large number of 
closely spaced images. As outlined in Section II-B, this 
makes fitting the entire image sequence almost completely 



automated and also provides significantly more data points in 
analyzing the activity. Finally, another tool has been added to 
calculate the contact area for each condyle between the 
femoral and polyethylene components. 

This paper is organized as follows: Section II describes 
the methodology behind the 3D analysis techniques, Section 

III presents experimental results obtained from contact area 
analysis including a comparison of our contact area technique 
versus actual wear from a polyethylene retrieval, and Section 

IV discusses and concludes. 

II. Methodology 
A. Direct 3D-to-2D registration 

The techniques employed in our different analytical tools 
are tailored to distinct applications and therefore largely vary 
from one to another. The 3D-to-2D registration method was 
developed previously [6,7]. A brief overview is provided here 
as background information for the extension of this method to 
video fitting. The basic setup used in 3D reconstruction of the 
fluoroscopy scene is shown in Fig. 1. Details of this method 
can be found in [6], including optimization techniques used, 
sensitivity to manual bias determined through range of 
convergence (ROC) testing, and the process of rendering a 
2D image of the implant from its 3D pose, dilating it, and 
correlating it against the original fluoroscopy image. 

Next, Fig. 2 shows a contour that has been manually 
traced around the femoral implant seen in the fluoroscopy 
image of Fig. 1. As discussed in [7], many 3D-to-2D 
registration techniques have been proposed which start by 
tracing a contour around the implant, similar to the one 
shown in Fig. 2. After extracting the contour, it is matched 
either against a template library [8,9] or by measuring the 
distance between the predicted and actual contours [10]. As 
discussed in [7], these techniques rely heavily on the 
Therefore, noise, occlusion, and poorly 
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video fitting process has been run on large image sequences 
over 200 images. The elimination of user interaction 
combined with the large amount of data received makes 
video fitting a useful extension to our basic 3D-to-2D 
q method. 



Fisj. 2. Contour manually [raced around femoral component. 



defined edges can largely degrade the performance of these 
methods. Our direct registration method correlates the 
predicted image to the original fluoroscopy image, 
eliminating the need to extract contours around the model 
silhouette. Consequently, direct registration performs better 
especially when dealing with noisy or occluded fluoroscopic 
images [7]. 

B. Video Fitting 

In order to perform the 3D-to-2D registration outlined in 
Section II-A, a starting pose for each model must be 
manually specified. When an activity such as a DKB is 
broken down into 10-15 fluoroscopic images, a starting 
position is manually specified to fit each image. Video fitting 
lis manual dependency by 
: sequence of closely spaced 
;o manually provide a starting 
its correct position for the first 
All subsequent images are 
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C. Contact Area 



Another tool added for 3D analysis is automatic 
calculation of contact area. This is most often used to find the 
contact area for each femoral condyle with the polyethylene 
component, although the user interface has been setup to 
allow contact area to be calculated between two arbitrary 
models. Fig. 4 shows a fluoroscopic image with the correctly 
fit femoral, tibial, and polyethylene components. Fig. 5 
shows the corresponding distance map and user interface 
used for contact area calculation. 

The polyethylene component in Fig. 4 is rigidly aligned to 
the top of the tibial component and is transformed to its 
correct position using the tibial component transformation. A 
distance map is then generated between the two selected 
models, as shown in Fig. 5. There can be some slight 
ambiguity in how "contact" is defined. It is initially defined 
as distances of +/- 1mm (negative distances indicating 
intersection of the models). The user interface allows this 
range to be changed, which is needed in some cases due to 
the approximation made by rigidly attaching the polyethylene 
insert to the tibial implant. 

III. Results 

The tools outlined in Section II are used primarily to 
obtain results for more clinical-based studies. Given that 
fact, coherently presenting results for the video fitting and 
contact area methodologies is somewhat difficult. The contact 



latically fit by using the final pose of the previous image 
as the initial pose for the next image. This concept is 
illustrated in Fig. 3. 

This example exaggerates the differences seen between 
consecutive images to highlight the video fitting process. 
The 3D-to-2D registration algorithm outlined in Section II-A 
is used to fit the 3D models to each fluoroscopy image given 
a starting pose. The final pose represents the pose which 
scored the highest using our direct registration method. The 
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Fig. 3. Exaggerated example of video fitting process. 




Distance map and user interface for contact area anal) sis. 
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area functionality described in Section II-C was used to 
calculate wear during DKB and gait activities for a wear 
retrieval patient. Before undergoing a revision, the patient 
performed both DKB and gait activities. Fig. 5 shows the 
contact area averaged over four discrete points along the 
DKB (0°,30 o ,60 o ,70° flexion) as well as the contact area 
during the heel strike of a gait cycle for a right kneed revision 
patient. Although there is significant overlap between the 
contact area during DKB and heel strike, some variation is 
seen especially on the lateral condyle. 

Next, the contact models in Fig. 6 were compared to the 
retrieval obtained after the patient underwent a revision 
surgery. Fig. 7a shows the retrieved polyethylene model with 
noticeable wear on both the medial and lateral condyles. 
Although wear occurs during both DKB and gait, in normal 
day-to-day situations gait activities happen substantially more 
than DKBs. The largest force endured by the knee during gait 
is at heel strike. Therefore it can be assumed that the wear 
seen during heel strike should correlate well with the overall 
wear of the polyethylene component. This claim is validated 
in Fig. 7b where the contact area shown in Fig. 6 for heel 
strike is overlayed on the actual polyethylene model shown in 
Fig. 7a. 

3 DKB - Average Heel Strike 




Fig. 6 Contact area averaged over four points along a DKB activity 
(0°,30 o ,60 o ,70° flexion) compared to contact area at the heel strike of a gait 
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IV. Conclusion 

The wear analysis presented in Section III highlights one 
application of the contact area functionality. In the future this 
contact information will be used as input for finite element 
models of the polyethylene. The goal is to do a finite element 
analysis of patients with a wide variety of implant types to 
gain a broad knowledge of how the different polyethylene 
components wear in vivo. Although results were not 
presented for the video fitting, work is currently being done 
to do a comparison between automated video fitting results 
and manually specifying the initial pose for each image in a 
300 image sequence. The end goal is to show a reduction in 
error due to the elimination of manual bias. Also, ROC 
testing was done in [6] on the convergence of the 3D-to-2D 
registration algorithm. The results show excellent 
convergence even for initial positions far away from the true 
pose (16mm in-plane translation, 128mm out-of-plane 
translation, and 16° of rotation around a randomly directed 
axis). The maximum displacement of the knee between 
frames when video is captured at 30 frames/sec for DKB and 
gait has been within this ROC for all video sequences 
analyzed, validating use of the final pose from a previous 
image as the initial pose for the following image. 

Fluoroscopy is a powerful tool for analysis of joints during 
dynamic, weight bearing activities. Two powerful new tools 
have been introduced that build on our fundamental 3D-to-2D 
registration method. Video fitting represents the next 
generation of 3D fluoroscopic analysis and will serve as a 
useful technique for more advanced 3D analysis techniques 
of fluoroscopy in the future. Calculation of contact area, 
combined with finite element methods, opens up new areas of 
analysis on both wear and the internal stress/strain 
characteristics of polyethylene components. 
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